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1 Introduction 

This guide is the culmination of a few months’ worth of blog posts. The subject is magic 
methods. 

What are magic methods? They’re everything in object-oriented Python. They’re special 
methods that you can define to add ’’magic” to your classes. They’re always surrounded by 

double underscores (e.g. init or It ). They’re also not as well documented as they 

need to be. All of the magic methods for Python appear in the same section in the Python 
docs, but they’re scattered about and only loosely organized. There’s hardly an example to be 
found in that section (and that may very well be by design, since they’re all detailed in the 
language reference, along with boring syntax descriptions, etc.). 

So, to fix what I perceived as a flaw in Python’s documentation, I set out to provide some 
more plain-English, example-driven documentation for Python’s magic methods. I started out 
with weekly blog posts, and now that Pve finished with those, Pve put together this guide. 

I hope you enjoy it. Use it as a tutorial, a refresher, or a reference; it’s just intended to be 
a user-friendly guide to Python’s magic methods. 

2 Construction and Initialization 

Everyone knows the most basic magic method, init . It’s the way that we can define the 

initialization behavior of an object. However, when I call x = SomeClass () , init is not 

the first thing to get called. Actually, it’s a method called new , which actually creates the 

instance, then passes any arguments at creation on to the initializer. At the other end of the 
object’s lifespan, there’s del . Let’s take a closer look at these 3 magic methods: 

new (els, [. . .) new is the first method to get called in an object’s instantiation. 

It takes the class, then any other arguments that it will pass along to init . - 

new is used fairly rarely, but it does have its purposes, particularly when subclassing 

an immutable type like a tuple or a string. I don’t want to go in to too much detail on 
new because it’s not too useful, but it is covered in great detail in the Python docs. 

init (self, [. . .) The initializer for the class. It gets passed whatever the primary con- 
structor was called with (so, for example, if we called x = SomeClass (10 , ’ foo’), 

_init would get passed 10 and ’ foo’ as arguments. init is almost universally 

used in Python class definitions. 

del (self) If new and init formed the constructor of the object, del is the 

destructor. It doesn’t implement behavior for the statement del x (so that code would 
not translate to x. del ()). Rather, it defines behavior for when an object is garbage 
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collected. It can be quite useful for objects that might require extra cleanup upon deletion, 

like sockets or file objects. Be careful, however, as there is no guarantee that del will 

be executed if the object is still alive when the interpreter exits, so del can’t serve 

as a replacement for good coding practices (like always closing a connection when you’re 

done with it. In fact, del should almost never be used because of the precarious 

circumstances under which it is called; use it with caution! 

Putting it all together, here’s an example of init and del in action: 

from os . path import join 
class FileObject: 

’’’Wrapper for file objects to make sure the file gets closed 
on deletion. ’ ’ ’ 

def __init __ ( self , f ilepath= ’ ~ ’ , f i lename =’ sample . txt ’) : 

# open a file filename in filepath in read and write mode 
self. file = open ( j oin ( f ilepath , filename), ’r+’) 

def __del__ ( self ) : 

self . file . close () 
del self . f ile 



3 Making Operators Work on Custom Classes 

One of the biggest advantages of using Python’s magic methods is that they provide a simple way 
to make objects behave like built-in types. That means you can avoid ugly, counter-intuitive, 
and nonstandard ways of performing basic operators. In some languages, it’s common to do 
something like this: 

if instance. equals (other_instance) : 

# do something 

You could certainly do this in Python, too, but this adds confusion and is unnecessarily verbose. 
Different libraries might use different names for the same operations, making the client do way 
more work than necessary. With the power of magic methods, however, we can define one 
method ( eq , in this case), and say what we mean instead: 

if instance == other_instance : 

#do something 

That’s part of the power of magic methods. The vast majority of them allow us to define 
meaning for operators so that we can use them on our own classes just like they were built in 
types. 

3.1 Comparison magic methods 

Python has a whole slew of magic methods designed to implement intuitive comparisons between 
objects using operators, not awkward method calls. They also provide a way to override the 
default Python behavior for comparisons of objects (by reference). Here’s the list of those 
methods and what they do: 

cmp (self, other) : cmp is the most basic of the comparison magic methods. It 

actually implements behavior for all of the comparison operators (j, ==, !=, etc.), but 
it might not do it the way you want (for example, if whether one instance was equal to 
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another were determined by one criterion and and whether an instance is greater than 

another were determined by something else). cmp should return a negative integer if 

self < other, zero if self == other, and positive if self > other. It’s usually best 

to define each comparison you need rather than define them all at once, but cmp - 

can be a good way to save repetition and improve clarity when you need all comparisons 
implemented with similar criteria. 

eq (self, other) Defines behavior for the equality operator, ==. 

ne (self, other) Defines behavior for the inequality operator, !=. 

It (self, other) Defines behavior for the less-than operator, <. 

gt (self, other) Defines behavior for the greater-than operator, >. 

le (self, other) Defines behavior for the less-than-or-equal-to operator, <=. 

ge (self, other) Defines behavior for the greater-than-or-equal-to operator, >=. 

For an example, consider a class to model a word. We might want to compare words 
lexicographically (by the alphabet), which is the default comparison behavior for strings, but 
we also might want to do it based on some other criterion, like length or number of syllables. 
In this example, we’ll compare by length. Here’s an implementation: 

class Word(str): 

’’’Class for words, defining comparison based on word length. ’’’ 
def __new__(cls, word): 

# Note that we have to use __new__ . This is because str is an 

# immutable type, so we have to initialize it early (at creation) 

if ’ ’ in word : 

print "Value contains spaces. Truncating to first space." 
word = word [: word . index ( ’ ’)] 

# Word is now all chars before first space 
return str . __new__ ( els , word) 
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Now, we can create two Words (by using Word( ’ f oo ’ ) and Word ( J bar ’ ) ) and compare them 

based on length. Note, however, that we didn’t define eq and ne . This is because 

this would lead to some weird behavior (notably that WordDfoo’) == Word (’bar’) would 
evaluate to true). It wouldn’t make sense to test for equality based on length, so we fall back 
on str’s implementation of equality. 

Now would be a good time to note that you don’t have to define every comparison magic 
method to get rich comparisons. The standard library has kindly provided us with a class 
decorator in the module functools that will define all rich comparison methods if you only 

define eq and one other (e.g. gt , It , etc.) This feature is only available in Python 

2.7, but when you get a chance it saves a great deal of time and effort. You can use it by placing 
@total_ordering above your class definition. 
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3.2 Numeric Magic Methods 

Just like you can create ways for instances of your class to be compared with comparison 
operators, you can define behavior for numeric operators. Buckle your seat belts, folks, there’s 
a lot of these. For organization’s sake, I’ve split the numeric magic methods into 5 categories: 
unary operators, normal arithmetic operators, reflected arithmetic operators (more on this 
later), augmented assignment, and type conversions. 

3.2.1 Unary operators and functions 

Unary operators and functions only have one operand, e.g. negation, absolute value, etc. 

pos (self) Implements behavior for unary positive (e.g. +some_object) 

neg (self) Implements behavior for negation (e.g. -some_object) 

abs (self) Implements behavior for the built in abs() function. 

invert (self) Implements behavior for inversion using the ~ operator. 

round (self, n) Implements behavior for the buil in round () function, n is the number 

of decimal places to round to. 

floor (self) : Implements behavior for math.floorO, i.e. , rounding down to the nearest 

integer. 

ceil (self) : Implements behavior for math.ceilO, i.e., rounding up to the nearest 

integer. 

trunc (self) : Implements behavior for math.truncO, i.e., truncating to an integral. 

3.3 Normal arithmetic operators 

Now, we cover the typical binary operators (and a function or two): +, -, * and the like. These 
are, for the most part, pretty self-explanatory. 

add (self, other) Implements addition. 

sub (self, other) Implements subtraction. 

mul (self, other) Implements multiplication. 

floordiv (self, other) Implements integer division using the // operator. 

div (self, other) Implements division using the / operator. 

truediv (self, other) Implements _true_ division. Note that this only works when from 

future import division is in effect. 

mod (self, other) Implements modulo using the °/ 0 operator. 

divmod (self, other) Implements behavior for long division using the divmodO built in 

function. 

pow Implements behavior for exponents using the ** operator. 
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lshift (self, other) Implements left bitwise shift using the << operator. 

rshift (self, other) Implements right bitwise shift using the » operator. 

and (self, other) Implements bitwise and using the & operator. 

or (self, other) Implements bitwise or using the | operator. 

xor (self, other) Implements bitwise xor using the ~ operator. 



3.3.1 Reflected arithmetic operators 

You know how I said I would get to reflected arithmetic in a bit? Some of you might think it’s 
some big, scary, foreign concept. It’s actually quite simple. Here’s an example: 

some_object + other 

That was ’’normal” addition. The reflected equivalent is the same thing, except with the 
operands switched around: 

other + some_object 

So, all of these magic methods do the same thing as their normal equivalents, except the 
perform the operation with other as the first operand and self as the second, rather than the 
other way around. In most cases, the result of a reflected operation is the same as its normal 

equivalent, so you may just end up defining radd as calling add and so on. Note 

that the object on the left hand side of the operator (other in the example) must not define 
(or return Notlmplemented) for its definition of the non-reflected version of an operation. For 

instance, in the example, some_object. radd will only be called if ‘other 1 does not define 

add . 



radd (self , 

rsub (self , 

rmul (self , 



other) Implements reflected addition, 
other) Implements reflected subtraction, 
other) Implements reflected multiplication. 



rfloordiv (self, other) Implements reflected integer division using the // operator. 



rdiv (self, other) Implements reflected division using the / operator. 

rtruediv (self, other) Implements reflected _true_ division. Note that this only works 

when from future import division is in effect. 

rmod (self, other) Implements reflected modulo using the % operator. 



rdivmod (self, other) Implements behavior for long division using the divmodO built 

in function, when divmod (other , self) is called. 



rpow Implements behavior for reflected exponents using the ** operator. 

rlshift (self, other) Implements reflected left bitwise shift using the << operator. 

rrshift (self, other) Implements reflected right bitwise shift using the >> operator. 

rand (self, other) Implements reflected bitwise and using the & operator. 

ror (self, other) Implements reflected bitwise or using the I operator. 

rxor (self, other) Implements reflected bitwise xor using the ~ operator. 
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3.3.2 Augmented assignment 

Python also has a wide variety of magic methods to allow custom behavior to be defined 
for augmented assignment. You’re probably already familiar with augmented assignment, it 
combines “normal” operators with assignment. If you still don’t know what I’m talking about, 
here’s an example: 
x = 5 

x += 1 # in other words x = x + 1 

Each of these methods should return the value that the variable on the left hand side should 

be assigned to (for instance, for a += b, iadd might return a + b, which would be assigned 

to a). Here’s the list: 

iadd (self, other) Implements addition with assignment. 

isub (self, other) Implements subtraction with assignment. 

imul (self, other) Implements multiplication with assignment. 

if loordiv (self , other) Implements integer division with assignment using the //= op- 
erator. 

idiv (self, other) Implements division with assignment using the /= operator. 

itruediv (self, other) Implements _true_ division with assignment. Note that this only 

works when from future import division is in effect. 

imod (self, other) Implements modulo with assignment using the %= operator. 

ipow Implements behavior for exponents with assignment using the **= operator. 

ilshift (self, other) Implements left bitwise shift with assignment using the <<= op- 
erator. 

irshift (self , other) Implements right bitwise shift with assignment using the »= op- 
erator. 

iand (self, other) Implements bitwise and with assignment using the &= operator. 

ior (self, other) Implements bitwise or with assignment using the I = operator. 

ixor (self, other) Implements bitwise xor with assignment using the ~= operator. 

3.3.3 Type conversion magic methods 

Python also has an array of magic methods designed to implement behavior for built in type 
conversion functions like float (). Here they are: 

int (self) Implements type conversion to int. 

long (self) Implements type conversion to long. 

float (self) Implements type conversion to float. 

complex (self) Implements type conversion to complex. 

oct (self) Implements type conversion to octal. 
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hex (self) Implements type conversion to hexadecimal. 

index (self) Implements type conversion to an int when the object is used in a slice 

expression. If you define a custom numeric type that might be used in slicing, you should 
define index . 

trunc (self) Called when math. trunc (self ) is called. trunc should return the 

value of self truncated to an integral type (usually a long). 

coerce (self, other) Method to implement mixed mode arithmetic. coerce should 

return None if type conversion is impossible. Otherwise, it should return a pair (2-tuple) 
of self and other, manipulated to have the same type. 



4 Representing your Classes 

It’s often useful to have a string representation of a class. In Python, there’s a few methods 

that you can implement in your class definition to customize how built in functions that return 

representations of your class behave. 

str (self) Defines behavior for when str() is called on an instance of your class. 

repr (self) Defines behavior for when reprQ is called on an instance of your class. The 

major difference between str() and repr() is intended audience. repr() is intended to 
produce output that is mostly machine-readable (in many cases, it could be valid Python 
code even), whereas str() is intended to be human-readable. 

Unicode (self ) Defines behavior for when Unicode () is called on an instance of your class. 

Unicode () is like str(), but it returns a Unicode string. Be wary: if a client calls str() 

on an instance of your class and you’ve only defined Unicode (), it won’t work. You 

should always try to define str () as well in case someone doesn’t have the luxury of 

using Unicode. 

format (self, formatstr) Defines behavior for when an instance of your class is used in 

new-style string formatting. For instance, "Hello, 0 : abc !". format (a) would lead to 

the call a. format ("abc"). This can be useful for defining your own numerical or 

string types that you might like to give special formatting options. 

hash (self) Defines behavior for when hashQ is called on an instance of your class. It 

has to return an integer, and its result is used for quick key comparison in dictionaries. 

Note that this usually entails implementing eq as well. Live by the following rule: a 

== b implies hash (a) == hash(b). 

nonzero (self) Defines behavior for when bool() is called on an instance of your class. 

Should return True or False, depending on whether you would want to consider the in- 
stance to be True or False. 

dir (self) : Defines behavior for when dir() is called on an instance of your class. This 

method should return a list of attributes for the user. Typically, implementing dir_- 

_ is unnecessary, but it can be vitally important for interactive use of your classes if you 

redefine getattr or getattribute (which you will see in the next section) or 

are otherwise dynamically generating attributes. 
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We’re pretty much done with the boring (and example-free) part of the magic methods guide. 
Now that we’ve covered some of the more basic magic methods, it’s time to move to more 
advanced material. 



5 Controlling Attribute Access 

Many people coming to Python from other languages complain that it lacks true encapsulation 
for classes (e.g. no way to define private attributes and then have public getter and setters). 
This couldn’t be farther than the truth: it just happens that Python accomplishes a great deal 
of encapsulation through “magic”, instead of explicit modifiers for methods or fields. Take a 
look: 

getattr (self, name) You can define behavior for when a user attempts to access an 

attribute that doesn’t exist (either at all or yet). This can be useful for catching and 
redirecting common misspellings, giving warnings about using deprecated attributes (you 
can still choose to compute and return that attribute, if you wish), or deftly handing an 
AttributeError. It only gets called when a nonexistent attribute is accessed, however, 
so it isn’t a true encapsulation solution. 

setattr (self, name, value) Unlike getattr , setattr is an encapsulation so- 
lution. It allows you to define behavior for assignment to an attribute regardless of whether 
or not that attribute exists, meaning you can define custom rules for any changes in the 

values of attributes. However, you have to be careful with how you use setattr , as 

the example at the end of the list will show. 

delattr (self , name) This is the exact same as setattr , but for deleting attributes 

instead of setting them. The same precautions need to be taken as with setattr as 

well in order to prevent infinite recursion (calling del self .name in the implementation 
of delattr would cause infinite recursion). 

getattribute (self, name) After all this, getattribute fits in pretty well with its 

companions setattr and delattr . However, I don’t recommend you use it. 

_getattribute can only be used with new-style classes (all classes are new-style in the 

newest versions of Python, and in older versions you can make a class new-style by sub- 
classing object. It allows you to define rules for whenever an attribute’s value is accessed. 
It suffers from some similar infinite recursion problems as its partners-in-crime (this time 

you call the base class’s getattribute method to prevent this). It also mainly 

obviates the need for getattr , which only gets called when getattribute is 

implemented if it is called explicitly or an AttributeError is raised. This method can be 
used (after all, it’s your choice), but I don’t recommend it because it has a small use case 
(it’s far more rare that we need special behavior to retrieve a value than to assign to it) 
and because it can be really difficult to implement bug-free. 

You can easily cause a problem in your definitions of any of the methods controlling attribute 
access. Consider this example: 

def __setattr__ ( self , name, value): 

self. name = value 

# since every time an attribute is assigned, __ s et at tr__ () 

# is called, this is recursion . So this really means 

# s el f . __ s et at tr_ _ ( ’ name ’ , value). Since the method keeps 

# calling itself, the recursion goes on forever causing a crash 
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def __setattr__ ( self , name, value): 

self . __dict__ [name] = value # assigning to the diet of names in the class 
# define custom behavior here 

Again, Python’s magic methods are incredibly powerful, and with great power comes great 
responsibility. It’s important to know the proper way to use magic methods so you don’t break 
any code. 

So, what have we learned about custom attribute access in Python? It’s not to be used 
lightly. In fact, it tends to be excessively powerful and counter-intuitive. But the reason why it 
exists is to scratch a certain itch: Python doesn’t seek to make bad things impossible, but just 
to make them difficult. Freedom is paramount, so you can really do whatever you want. Here’s 
an example of some of the special attribute access methods in action (note that we use super 
because not all classes have an attribute diet ): 

class AccessCounter (object ) : 

’’’A class that contains a value and implements an 
access counter. The counter increments each time the 
value is changed. ’’’ 

def __init __ ( self , val): 

super ( AccessCounter , self). setattr__(’ counter 1 , 0) 

super ( AccessCounter , self ). __setattr__ (’ value ’ , val) 

def __setattr__ ( self , name, value): 
if name == ’value’: 

super ( AccessCounter , self ) . __setattr._ ( ’ counter ’ , 

self. counter + 1) 

# Make this unconditional . 

# If you want to prevent other attributes to be set, 

# raise AttributeError (name) 

super ( AccessCounter , self ). __setattr__ (name , value) 

def __delattr__ ( self , name): 
if name == ’value’: 

super ( AccessCounter , self ) . __setattr._ ( ’ counter ’ , 

self. counter + 1) 

super ( AccessCounter , self ) . __delattr__ (name)] 



6 Making Custom Sequences 

There’s a number of ways to get your Python classes to act like built in sequences (diet, tuple, 
list, string, etc.). These are by far my favorite magic methods in Python because of the absurd 
degree of control they give you and the way that they magically make a whole array of global 
functions work beautifully on instances of your class. But before we get down to the good stuff, 
a quick word on requirements. 

6.1 Requirements 

Now that we’re talking about creating your own sequences in Python, it’s time to talk about 
_protocols_. Protocols are somewhat similar to interfaces in other languages in that they give 
you a set of methods you must define. However, in Python protocols are totally informal and 
require no explicit declarations to implement. Rather, they’re more like guidelines. 

Why are we talking about protocols now? Because implementing custom container types in 
Python involves using some of these protocols. First, there’s the protocol for defining immutable 
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containers: to make an immutable container, you need only define len and getitem_- 

(more on these later). The mutable container protocol requires everything that immutable 

containers require plus setitem and delitem . Lastly, if you want your object to be 

iterable, you'll have to define iter , which returns an iterator. That iterator must conform 

to an iterator protocol, which requires iterators to have methods called iter (returning 

itself) and next. 

6.2 The magic behind containers 

len (self) Returns the length of the container. Part of the protocol for both immutable 

and mutable containers. 

getitem (self, key) Defines behavior for when an item is accessed, using the notation 

self [key] . This is also part of both the mutable and immutable container protocols. It 
should also raise appropriate exceptions: TypeError if the type of the key is wrong and 
KeyError if there is no corresponding value for the key. 

setitem (self , key, value) Defines behavior for when an item is assigned to, using the 

notation self [key] = value. This is part of the mutable container protocol. Again, you 
should raise KeyError and TypeError where appropriate. 

delitem (self , key) Defines behavior for when an item is deleted (e.g. del self [key] ). 

This is only part of the mutable container protocol. You must raise the appropriate 
exceptions when an invalid key is used. 

iter (self) Should return an iterator for the container. Iterators are returned in a number 

of contexts, most notably by the iterQ built in function and when a container is looped 
over using the form for x in container : . Iterators are their own objects, and they also 
must define an iter method that returns self. 

reversed (self) Called to implement behavior for the reversedO built in function. 

Should return a reversed version of the sequence. Implement this only if the sequence 
class is ordered, like list or tuple. 

contains (self, item) contains defines behavior for membership tests using in 

and not in. Why isn’t this part of a sequence protocol, you ask? Because when - 

contains isn’t defined, Python just iterates over the sequence and returns True if it 

comes across the item it’s looking for. 

missing (self, key) missing is used in subclasses of diet. It defines behavior for 

whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had 

a dictionary d and said d["george"] when "george" is not a key in the diet, d. - 

missing ("george") would be called). 

6.3 An example 

For our example, let’s look at a list that implements some functional constructs that you might 
be used to from other languages (Haskell, for example): 

class Funct i onalLi st : 

’’’A class wrapping a list with some extra functional 
magic, like head, tail, init , last, drop, and take. ’’’ 

def __init __ ( self , values = None ) : 



10 




if values is None: 

self . values = [] 

else : 

self. values = values 

def __len__ ( self ) : 

return len ( self . values ) 

def __getitem__ ( self , key): 

# if key is of invalid type or value, the list values 

# will raise the error 
return self . values [key] 

def __ set it em__ ( self , key, value): 
self . values [key] = value 

def __delitem__ ( self , key): 
del self . values [key ] 

def __iter__ ( self ) : 

return iter ( self . values ) 

def __r ever sed__ ( self ) : 

return reversed ( self . values ) 

def append(self , value): 

self . values . append(value) 
def head ( self ) : 

# get the first element 
return self . values [0] 

def tail ( self ) : 

# get all elements after the first 
return self . values [1 : ] 

def init ( self ) : 

# get elements up to the last 
return self . values [: -1] 

def last ( self ) : 

# get last element 
return self . values [-1] 

def drop(self , n): 

# get all elements except first n 
return self . values [n : ] 

def takeCself , n) : 

# get first n elements 
return self . values [: n] 

There you have it, a (marginally) useful example of how to implement your own sequence. 

Of course, there are more useful applications of custom sequences, but quite a few of them 
are already implemented in the standard library (batteries included, right?), like Counter, 
OrderedDict, and NamedTuple. 

7 Reflection 

You can also control how reflection works using the built in functions isinstance and issubclass ( ) 
behaves by defining magic methods. The magic methods are: 

instancecheck (self, instance) Checks if an instance is an instance of the class you 

defined (e.g. isinstance (instance , class). 
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subclasscheck (self, subclass) Checks if a class subclasses the class you defined (e.g. 

issubclass (subclass , class)). 

The use case for these magic methods might seem small, and that may very well be true. 
I won’t spend too much more time on reflection magic methods because they aren’t very im- 
portant, but they reflect something important about object-oriented programming in Python 
and Python in general: there is almost always an easy way to do something, even if it’s rarely 
necessary. These magic methods might not seem useful, but if you ever need them you’ll be 
glad that they’re there (and that you read this guide!). 

8 Abstract Base Classes 

See http:/ /docs. python. org/2/library/abc.html. 

9 Callable Objects 

As you may already know, in Python, functions are first-class objects. This means that they 
can be passed to functions and methods just as if they were objects of any other kind. This is 
an incredibly powerful feature. 

A special magic method in Python allows instances of your classes to behave as if they were 
functions, so that you can “call” them, pass them to functions that take functions as arguments, 
and so on. This is another powerful convenience feature that makes programming in Python 
that much sweeter. 

call (self, [args . . .]) Allows an instance of a class to be called as a function. Essen- 
tially, this means that x() is the same as x. call (). Note that call takes a 

variable number of arguments; this means that you define call as you would any 

other function, taking however many arguments you’d like it to. 

call can be particularly useful in classes whose instances that need to often change 

state. “Calling” the instance can be an intuitive and elegant way to change the object’s state. 
An example might be a class representing an entity’s position on a plane: 

class Entity : 

’’’Class to represent an entity. Callable to update 
the entity’s position . ’’’ 

def __init __ ( self , size, x, y): 
self.x, self.y = x, y 
self . size = size 

def __call__ ( self , x, y): 

’’’Change the position of the entity. ’’’ 
self.x, self.y = x, y 

# snip . . . 



10 Context Managers 

In Python 2.5, a new keyword was introduced in Python along with a new method for code 
reuse, the with statement. The concept of context managers was hardly new in Python (it was 
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implemented before as a part of the library), but not until PEP 343 was accepted did it achieve 
status as a first class language construct. You may have seen with statements before: 

with open ( ’ f oo . txt ’ ) as bar: 

# perform some action with bar 

Context managers allow setup and cleanup actions to be taken for objects when their creation 
is wrapped with a with statement. The behavior of the context manager is determined by two 
magic methods: 

enter (self) Defines what the context manager should do at the beginning of the block 

created by the with statement. Note that the return value of enter is bound to the 

target of the with statement, or the name after the as. 

exit (self, exception_type , exception_value , traceback) Defines what the context 

manager should do after its block has been executed (or terminates). It can be used to 
handle exceptions, perform cleanup, or do something always done immediately after the 
action in the block. If the block executes successfully, exception_type, exception- 
value, and traceback will be None. Otherwise, you can choose to handle the exception 

or let the user handle it; if you want to handle it, make sure exit returns True 

after all is said and done. If you don’t want the exception to be handled by the context 
manager, just let it happen. 

enter and exit can be useful for specific classes that have well-defined and com- 
mon behavior for setup and cleanup. You can also use these methods to create generic context 
managers that wrap other objects. Here’s an example: 

class Closer : 

’’’A context manager to automatically close an object with a 
closeO method in a with statement . ’’’ 

def __init __ ( self , obj): 
self . obj = obj 

def __enter__ ( self ) : 

return self . obj # bound to target 

def __exit __ ( self , except ion_type , exception_val , trace): 
try : 

self . obj . close () 

except AttributeError : # obj isn’t closable 

print ’Not closable . ’ 

return True # exception handled successfully 

Here’s an example of Closer in action, using an FTP connection to demonstrate it (a closable 
socket): 

>>> from magicmethods import Closer 
>>> from ftplib import FTP 

>>> with Clo ser ( FTP ( ’ f tp . some s i t e . com ’ ) ) as conn: 

. . . conn . dir ( ) 

# output omitted for brevity 
>>> conn . dir ( ) 

# long AttributeError message, can’t use a connection that’s closed 
>>> with Closer ( int (5) ) as i: 

... i += 1 

Not closable . 
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>>> i 
6 

See how our wrapper gracefully handled both proper and improper uses? That’s the power of 
context managers and magic methods. Note that the Python standard library includes a module 
contextlib that contains a context manager, contextlib . closingQ , that does approximately 
the same thing (without any handling of the case where an object does not have a close () 
method) . 

11 Building Descriptor Objects 

Descriptors are classes which, when accessed through either getting, setting, or deleting, can also 
alter other objects. Descriptors aren’t meant to stand alone; rather, they’re meant to be held 
by an owner class. Descriptors can be useful when building object-oriented databases or classes 
that have attributes whose values are dependent on each other. Descriptors are particularly 
useful when representing attributes in several different units of measurement or representing 
computed attributes (like distance from the origin in a class to represent a point on a grid). 

To be a descriptor, a class must have at least one of get , set , and delete - 

implemented. 

Let’s take a look at those magic methods: 

get (self, instance, owner) Define behavior for when the descriptor’s value is retrieved. 

instance is the instance of the owner object, owner is the owner class itself. 

set (self , instance , value) Define behavior for when the descriptor’s value is changed. 

instance is the instance of the owner class and value is the value to set the descriptor 
to. 

delete (self, instance) Define behavior for when the descriptor’s value is deleted, instance 

is the instance of the owner object. 

Now, an example of a useful application of descriptors: unit conversions. 

class Meter ( obj ect ) : 

’’’Descriptor for a meter. ’’’ 

def __init __ ( self , value=0.0): 

self . value = float (value) 
def __get__ ( self , instance, owner): 
return self . value 

def __set__ ( self , instance, value): 
self . value = float (value) 

class Foot (obj ect ) : 

’’’Descriptor for a foot. ’’’ 

def __get__ ( self , instance, owner): 
return instance . meter * 3.2808 
def __set__ ( self , instance, value): 

instance . meter = float (value) / 3.2808 

class Distance (obj ect ) : 

’’’Class to represent distance holding two descriptors for feet and 
meters . ’ ’ ’ 
meter = Meter () 
foot = Foot () 
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12 Copying 



Sometimes, particularly when dealing with mutable objects, you want to be able to copy an 
object and make changes without affecting what you copied from. This is where Python’s copy 
comes into play. However (fortunately), Python modules are not sentient, so we don’t have to 
worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy 
things. 

copy (self) : Defines behavior for copy . copy () for instances of your class, copy, copy () 

returns a .shallow copy, of your object - this means that, while the instance itself is a new in- 
stance, all of its data is referenced - i.e. , the object itself is copied, but its data is still referenced 
(and hence changes to data in a shallow copy may cause changes in the original). 

deepcopy (self, memodict=) : Defines behavior for copy . deep copy () for instances of 

your class, copy. deepcopy () returns a .deep copy, of your object - the object .and. its data 
are both copied, memodict is a cache of previously copied objects - this optimizes copying and 
prevents infinite recursion when copying recursive data structures. When you want to deep 
copy an individual attribute, call copy . deepcopy () on that attribute with memodict as the 
first argument. 

What are some use cases for these magic methods? As always, in any case where you need 
more fine-grained control than what the default behavior gives you. For instance, if you are 
attempting to copy an object that stores a cache as a dictionary (which might be large), it 
might not make sense to copy the cache as well - if the cache can be shared in memory between 
instances, then it should be. 

13 Pickling 

If you spend time with other Pythonistas, chances are you’ve at least heard of pickling. Pickling 
is a serialization process for Python data structures, and can be incredibly useful when you need 
to store an object and retrieve it later. It’s also a major source of worries and confusion. 

Pickling is so important that it doesn’t just have its own module (pickle), but its own 
protocol and the magic methods to go with it. But first, a brief word on how to pickle existing 
types(feel free to skip it if you already know). 

13.1 Pickling: A Quick Soak in the Brine 

Let’s dive into pickling. Say you have a dictionary that you want to store and retrieve later. 
You could write it’s contents to a file, carefully making sure that you write correct syntax, then 
retrieve it using either exec() or processing the file input. But this is precarious at best: if you 
store important data in plain text, it could be corrupted or changed in any number of ways to 
make your program crash or worse run malicious code on your computer. Instead, we’re going 



to pickle it: 




import pickle 


data = { ’ f oo ’ : 


1 — 1 
to 

CO 


’ bar ’ : 


( ’ Hello ’ , 


’ baz ’ : 


True } 



jar = open (’ data . pkl ’ , ’ wb ’ ) 

pickle . dump (data , jar) # write the pickled data to the file jar 
j ar . close () 

Now, a few hours later, we want it back. All we have to do is unpickle it: 
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import pickle 



pkl_file = open ( ’ dat a . pkl ’ , ’ rb ’ ) # connect to the pickled data 

data = pi ckle . load ( pkl_f ile ) # load it into a variable 

print data 
pkl_file . close () 

What happens? Exactly what you expect. It’s just like we had data all along. 

Now, for a word of caution: pickling is not perfect. Pickle files are easily corrupted on 
accident and on purpose. Pickling may be more secure than using flat text files, but it still can 
be used to run malicious code. It’s also incompatible across versions of Python, so don’t expect 
to distribute pickled objects and expect people to be able to open them. However, it can also 
be a powerful tool for caching and other common serialization tasks. 

13.2 Pickling your own Objects 

Pickling isn’t just for built-in types. It’s for any class that follows the pickle protocol. The 
pickle protocol has four optional methods for Python objects to customize how they act (it’s a 
bit different for C extensions, but that’s not in our scope): 

getinitargs (self) If you’d like for init to be called when your class is unpickled, 

you can define getinitargs , which should return a tuple of the arguments that you’d 

like to be passed to init . Note that this method will only work for old-style classes. 

getnewargs (self) For new-style classes, you can influence what arguments get passed to 

new upon unpickling. This method should also return a tuple of arguments that will 

then be passed to new . 

getstate (self) Instead of the object’s diet attribute being stored, you can return 

a custom state to be stored when the object is pickled. That state will be used by - 

setstate when the object is unpickled. 

setstate (self, state) When the object is unpickled, if setstate is defined the 

object’s state will be passed to it instead of directly applied to the object’s diet . 

This goes hand in hand with getstate : when both are defined, you can represent 

the object’s pickled state however you want with whatever you want. 

reduce (self) When defining extension types (i.e., types implemented using Python’s C 

API), you have to tell Python how to pickle them if you want them to pickle them. - 

reduce () is called when an object defining it is pickled. It can either return a string 

representing a global name that Python will look up and pickle, or a tuple. The tuple 
contains between 2 and 5 elements: a callable object that is called to recreate the object, a 

tuple of arguments for that callable object, state to be passed to setstate (optional), 

an iterator yielding list items to be pickled (optional), and an iterator yielding dictionary 
items to be pickled (optional). 

reduce_ex (self) reduce_ex exists for compatibility. If it is defined, reduce_- 

ex will be called over reduce on pickling. reduce can be defined as well for 

older versions of the pickling API that did not support reduce_ex . 
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13.3 An Example 

Our example is a Slate, which remembers what its values have been and when those values 
were written to it. However, this particular slate goes blank each time it is pickled: the current 
value will not be saved. 

import time 
class Slate : 

’’’Class to store a string and a changelog, and forget its value when 
pickled. ’ ’ ’ 

def __init __ ( self , value): 
self. value = value 

self . last_change = t ime . asct ime ( ) 
self. history = {} 

def change (self , new_value): 

# Change the value. Commit last value to history 
self . history [self . last_change] = self . value 
self. value = new_value 

self . last.change = time . asctime () 

def print_changes ( self ) : 

print ’Changelog for Slate object: ’ 
for k, v in self . history . items () : 
print ’7 0 s\t '/.s’ '/, (k, v) 

def __get st at e__ ( self ) : 

# Deliberately do not return self .value or self. last_change. 

# We want to have a "blank slate" when we unpickle. 
return self .history 

def __ s et st at e__ ( self , state): 

# Make self . history = state and last_change and value undefined 
self. history = state 

self. value, self . last_change = None, None 



14 Conclusion 

The goal of this guide is to bring something to anyone that reads it, regardless of their experi- 
ence with Python or object-oriented programming. If you’re just getting started with Python, 
you’ve gained valuable knowledge of the basics of writing feature-rich, elegant, and easy-to-use 
classes. If you’re an intermediate Python programmer, you’ve probably picked up some slick 
new concepts and strategies and some good ways to reduce the amount of code written by you 
and clients. If you’re an expert Pythonista, you’ve been refreshed on some of the stuff you 
might have forgotten about and maybe picked up a few new tricks along the way. Whatever 
your experience level, I hope that this trip through Python’s special methods has been truly 
magical (I couldn’t resist the final pun). 
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15 Appendix 1: How to Call Magic Methods 



Some of the magic methods in Python directly map to built-in functions; in this case, how to 
invoke them is fairly obvious. However, in other cases, the invocation is far less obvious. This 
appendix is devoted to exposing non-obvious syntax that leads to magic methods getting called. 



Magic Method 


When it gets invoked 


Explanation 


new (els [,...]) 


instance = MyClass (argl , 
arg2) 


new is called on instance 

creation 


init (self [,...]) 


instance = MyClass (argl , 
arg2) 


init is called on instance 

creation 


emp (self, other) 


self == other, self > 

other, etc. 


Called for any comparison 


pos (self) 


+self 


Unary plus sign 


neg (self) 


-self 


Unary minus sign 


invert (self) 


self 


Bitwise inversion 


index (self) 


x [self] 


Conversion when object is 
used as index 


nonzero (self) 


bool(self ) 


Boolean value of the object 


getattr (self, name) 


self. name # name doesn’t 
exist 


Accessing nonexistent at- 
tribute 


setattr (self, name, 

val) 


self. name = val 


Assigning to an attribute 


delattr (self, name) 


del self. name 


Deleting an attribute 


getattribute (self , 

name) 


self .name 


Accessing any attribute 


getitem (self, key) 


self [key] 


Accessing an item using an in- 
dex 


setitem (self, key, 

val) 


self [key] = val 


Assigning to an item using an 
index 


delitem (self, key) 


del self [key] 


Deleting an item using an in- 
dex 


iter (self) 


for x in self 


Iteration 


contains (self , 

value) 


value in self, value not 
in self 


Membership tests using in 


call (self [,...]) 


self (args) 


’’Calling” an instance 


enter (self) 


with self as x: 


with statement context man- 
agers 


exit (self, exc, val, 

trace) 


with self as x: 


with statement context man- 
agers 


get state (self) 


pickle . dump (pkl_f ile , 
self) 


Pickling 


set state (self) 


data = pickle . load(pkl_- 
f ile) 


Pickling 
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16 Appendix 2: Changes in Python 3 



Here, we document a few major places where Python 3 differs from 2.x in terms of its object 
model: 

• Since the distinction between string and Unicode has been done away with in Python 

3, Unicode is gone and bytes (which behaves similarly to str and - 

Unicode in 2.7) exists for a new built-in for constructing byte arrays. 

• Since division defaults to true division in Python 3, div is gone in Python 3 

• coerce is gone due to redundancy with other magic methods and confusing behavior 

• cmp is gone due to redundancy with other magic methods 

• nonzero has been renamed to bool 
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